Implementing lightweight detection for new write operations #52977
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
alrod
requested review from
amnguye,
fabiocav,
jalauzon-msft,
jaschrep-msft,
mathewc,
nickliu-msft,
seanmcc-msft and
tg-msft
as code owners
There was a problem hiding this comment.
Pull Request Overview
This PR implements lightweight detection for write operations to optimize the scaling logic performance. Instead of parsing all logs to extract blob paths, it only checks for the presence of write operations to trigger scaling events.
- Adds a new
HasBlobWritesAsyncmethod that efficiently detects write operations without full log parsing - Updates the scaling monitor to use the lightweight detection method
- Removes unused test field and improves logging messages
Reviewed Changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| BlobLogListener.cs | Adds new HasBlobWritesAsync method for lightweight write operation detection |
| BlobScalerMonitorProvider.cs | Updates scaling logic to use the new detection method and removes unused field |
| BlobLogListenerTests.cs | Adds comprehensive test coverage for the new HasBlobWritesAsync method |
.../Microsoft.Azure.WebJobs.Extensions.Storage.Blobs/src/Listeners/BlobScalerMonitorProvider.cs
Outdated
Show resolved
Hide resolved
...age/Microsoft.Azure.WebJobs.Extensions.Storage.Blobs/tests/Listeners/BlobLogListenerTests.cs
Outdated
Show resolved
Hide resolved
...age/Microsoft.Azure.WebJobs.Extensions.Storage.Blobs/tests/Listeners/BlobLogListenerTests.cs
Show resolved
Hide resolved
amnguye
reviewed
Oct 3, 2025
sdk/storage/Microsoft.Azure.WebJobs.Extensions.Storage.Blobs/src/Listeners/BlobLogListener.cs
Outdated
Show resolved
Hide resolved
amnguye
reviewed
Oct 3, 2025
.../Microsoft.Azure.WebJobs.Extensions.Storage.Blobs/src/Listeners/BlobScalerMonitorProvider.cs
Show resolved
Hide resolved
amnguye
reviewed
Oct 3, 2025
alrod
force-pushed
the
alrod/lighweight-new-writes-detection
branch
from
7404367 to
70d5716
Compare
amnguye
approved these changes
Oct 8, 2025
|
@amnguye Nobody from the Functions side has yet had a chance to review this, and I see you just merged this? |
amnguye
added a commit
that referenced
this pull request
Oct 8, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR introduces a streamlined approach to zero-to-one scaling logic within the Azure SDK. The current implementation reads and parses all classic monitoring logs in the $log container to extract modified blob paths. However, for scale-out from zero to one, this level of detail is unnecessary.
The updated logic simplifies the process by checking only for the presence of write operations within the last two hours—without downloading or parsing the full log content. This optimization reduces memory consumption and improves performance, especially in environments with high write volumes.